Image processing apparatus wirelessly communicating with external device, electronic device, control method thereof, and recording medium

ABSTRACT

An image processing apparatus that communicates with an external device and combines a sub image received from the external device with a main image includes a determination unit that determines a wipe shape of the sub image, and a notification unit that notifies the external device of the determined wipe shape.

BACKGROUND Field

The present disclosure relates to an image processing apparatuswirelessly communicating with an external device, an electronic device,a control method thereof, and a recording medium.

Description of the Related Art

Some imaging apparatuses such as digital cameras have a wirelesscommunication function to perform transmission and reception of imagedata with external devices. In addition, some imaging apparatuses have afunction of receiving an image captured by an external device andcombining it with an image captured by itself. For example, JapanesePatent Laid-Open No. 6-165029 discloses a technique of combining a firstvideo signal as an output of a first imaging unit and a second videosignal as an output of a second imaging unit to than a third videosignal.

In the conventional technique disclosed in Japanese Patent Laid-Open No.6-165029, a sub image is superimposed on a main image and displayed in aseparate screen like a small window (referred to as a wipe in thepresent application).

Japanese Patent Laid-Open No. 6-165029 is not seen to discuss the wipeshape, and the sub image is always displayed in the wipe of the sameshape. That is, there is no description of changing the wipe to adifferent shape.

SUMMARY

According to an aspect of the present disclosure, an image processingapparatus that communicates with an external device and combines a subimage received from the external device with a main image includes adetermination unit configured to determine a wipe shape of the subimage, and a notification unit configured to notify the external deviceof the determined wipe shape.

Further features will become apparent from the following description ofexemplary embodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A to 1C are diagrams illustrating a configuration example of adigital camera according to a first exemplary embodiment.

FIG. 2 is a diagram illustrating a configuration example of a smartdevice according to the first exemplary embodiment.

FIG. 3 is a sequence diagram illustrating processing by the digitalcamera and the smart device according to the first exemplary embodiment.

FIGS. 4A and 4B are diagrams illustrating examples of screens displayedon a display unit of the digital camera and a display unit of the smartdevice according to the first exemplary embodiment.

FIG. 5A is a flowchart describing the operations of the digital cameraaccording to the first exemplary embodiment.

FIG. 5B is a flowchart describing details of a wipe shape determinationprocessing according to the first exemplary embodiment.

FIG. 6 is a sequence diagram illustrating processing by a digital cameraand a smart device according to a second exemplary embodiment.

FIGS. 7A to 7C are diagrams illustrating examples of screens displayedon a display unit of the digital camera and a display unit of the smartdevice according to the second exemplary embodiment.

FIG. 8 is a flowchart describing the operations of the digital cameraaccording to the second exemplary embodiment.

FIG. 9A is a flowchart describing the operations of the smart deviceaccording to the second exemplary embodiment.

FIG. 9B is a flowchart describing details of a wipe shape determinationprocessing according to the second exemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments will be described with reference to theaccompanying drawings.

First Exemplary Embodiment [Configuration of a Digital Camera 100]

FIGS. 1A to 1C are diagrams illustrating a configuration example of thedigital camera 100 according to the present exemplary embodiment. Thedigital camera 100 is an example of an image processing apparatusaccording to the exemplary embodiment.

FIG. 1A is a block diagram illustrating a configuration example of thedigital camera 100.

A control unit 101 controls individual units of the digital camera 100according to input signals and programs. The control unit 101 may notcontrol entire apparatus but a plurality of pieces of hardware may shareprocessing to control the entire apparatus.

An imaging unit 102 includes, for example, an optical system thatcontrols an optical lens unit, diaphragm, zoom, and focus, an imagingelement that converts light (video image) introduced through the opticallens unit into an electrical video signal, and others. In general, theimaging element is a complementary metal oxide semiconductor (CMOS) or acharge coupled device (CCD). Under the control of the control unit 101,the imaging unit 102 converts subject light imaged by a lens included inthe imaging unit 102 into an electric signal by the imaging element,subjects the electric signal to noise reduction processing, and outputsthe digital data as image data. The image data is recorded on arecording medium 110 according to the standards of design rule forcamera file (DCF) system, for example, or is combined with another imageby an image composition unit 112.

A non-volatile memory 103 is a non-volatile memory that can electricallyerase and record, and stores programs to be executed by the control unit101.

A work memory 104 is used as a buffer memory that temporarily holdsimage data captured by the imaging unit 102, an image display memory fora display unit 106, a work area for the control unit 101, and the like.

An operation unit 105 is used to receive an instruction from the user tothe digital camera 100. The operation unit 105 includes, for example, apower button for instructing powering on/off of the digital camera 100,a release switch for instructing photographing, and a reproductionbutton for instructing reproduction of image data. The operation unit105 also includes a connection button for starting communication with anexternal device via a connection unit 111. The operation unit 105 alsoincludes a touch panel formed on the display unit 106. The releaseswitch has a SW 1 and a SW 2. When the release switch becomes in ahalf-stroke state, the SW 1 is turned on. Accordingly, instructions forpreparing for photographing such as autofocus (AF) processing, automaticexposure (AE) processing, auto white balance (AWB) processing, andpre-flash (EF) processing are accepted. When the release switch becomesin a full-stroke state, the SW 2 is turned on. Accordingly, aninstruction for photographing is accepted.

The display unit 106 displays a viewfinder image at the time ofphotographing, captured image data, characters for interactiveoperations, and others. The digital camera 100 does not necessarily needto include the display unit 106. The digital camera 100 has merely adisplay control function of connecting to the internal or externaldisplay unit 106 to control display on the display unit 106.

The recording medium 110 records the image data output from the imagingunit 102 and the image data combined by the image composition unit 112.The recording medium 110 may be detachably attached to the digitalcamera 100 or may be configured to be built in the digital camera 100.That is, the digital camera 100 has merely a function of accessing therecording medium 110.

The connection unit 111 is an interface for connecting to an externaldevice. The digital camera 100 can exchange data with an external devicevia the connection unit 111. For example, the digital camera 100 canreceive image data generated by an external device from the externaldevice via the connection unit 111. The connection unit 111 includes,for example, an interface for communicating with an external device in awireless LAN according to the IEEE 802.11 standard. The control unit 101implements wireless communication with an external device by controllingthe connection unit 111. The communication method is not limited to thewireless LAN, and other wireless communication methods such as aninfrared communication method, Bluetooth®, wireless USB or the like maybe used.

The image composition unit 112 can combine two or more images. Forexample, the image composition unit 112 combines the image output fromthe imaging unit 102 and the image received from an external device viathe connection unit 111 to generate and output one image data.Specifically, assuming that the image output from the imaging unit 102is a main image and the image received from the external device is asub-image, the image composition unit 112 can superimpose the sub imageon the main image so that the sub image is displayed in a wipe. Theimage data output from the image composition unit 112 is recorded on therecording medium 110 according to the DCF standard.

The digital camera 100 according to the present exemplary embodiment hasan AP mode in which to operate as an access point in an infrastructuremode and a CL mode in which to operate as a client in the infrastructuremode. By operating the connection unit 111 in the CL mode under thecontrol of the control unit 101, the digital camera 100 can operate as aCL device in the infrastructure mode. When the digital camera 100operates as a CL device, the digital camera 100 can connect to aperipheral AP device participate in the network formed by the AP device.By operating the connection unit 111 in the AP mode under the control ofthe control unit 101, the digital camera 100 can operate as a kind of APbut as a simple AP with more limited functions (hereinafter referred toas “simple AP”). When the digital camera 100 operates as a simple AP,the digital camera 100 forms a network by itself. Peripheral devices ofthe digital camera 100 can recognize the digital camera 100 as an APdevice and participate in the network formed by the digital camera 100.Programs for operating the digital camera 100 in this way are held inthe non-volatile memory 103. Although the digital camera 100 accordingto the present exemplary embodiment is a type of AP but a simple AP thatdoes not have a gateway function of transferring data received from a CLdevice to an Internet provider or the like. Therefore, even whenreceiving data from peripheral devices participating in the networkformed by itself, the digital camera 100 cannot transfer the data to anetwork such as the Internet.

Next, FIGS. 1B and 1C are diagrams illustrating an external appearanceexample of the digital camera 100. A release switch 105 a, areproduction button 105 b, a direction key 105 c, a touch panel 105 d,and others are examples of operation members constituting the operationunit 105 and are disposed at appropriate positions of the digital camera100. The display unit 106 is disposed on the back of the digital camera100.

The digital camera 100 has been described as an example of the imageprocessing apparatus in the present exemplary embodiment, but the imageprocessing apparatus is not limited thereto. For example, the imageprocessing apparatus can be an information processing apparatus such asa mobile phone, a portable media player, a tablet device, or a personalcomputer.

[Configuration of a Smart Device 200]

FIG. 2 is a diagram illustrating a configuration example of a smartdevice 200 according to the present exemplary embodiment. The smartdevice 200 is an example of an electronic device in the presentexemplary embodiment, and serves as an external device for the digitalcamera 100. The smart device includes a mobile phone such as asmartphone or a tablet device.

A control unit 201 controls individual units of the smart device 200according to input signals and programs. The control unit 201 may notcontrol the entire apparatus but a plurality of pieces of hardware mayshare processing to control the entire apparatus.

Under the control of the control unit 201, an imaging unit 202 convertssubject light imaged by a lens included in the imaging unit 202 into anelectric signal by an imaging element, subjects the electric signal tonoise reduction processing, and outputs the digital data as image data.After the image data is stored in a buffer memory, the control unit 201executes a predetermined calculation and records the image data on arecording medium 210 or transmits the image data to an external devicevia a connection unit 211.

A non-volatile memory 203 is a non-volatile memory that can electricallyerase and record. The non-volatile memory 203 records an operatingsystem (OS) as basic software to be executed by the control unit 201 andapplications which implement applicative functions in cooperation withthis OS. The non-volatile memory 203 also stores an application forcommunicating with the digital camera 100.

A work memory 204 is used as an image display memory for the displayunit 206, a work area for the control unit 201, and the like.

An operation unit 205 is used to receive instructions from the user tothe smart device 200. The operation unit 205 includes, for example, apower button for instructing powering on/off of the smart device 200,and a touch panel formed on a display unit 206.

The display unit 206 displays image data, characters for interactiveoperations, and the like. The smart device 200 does not necessarily needto include the display unit 206. The smart device 200 has merely adisplay control function of connecting to the internal or externaldisplay unit 206 and controlling display on the display unit 206.

The recording medium 210 records the image data output from the imagingunit 202. The recording medium 210 may be detachably attached to thesmart device 200 or may be built in the smart device 200. That is, thesmart device 200 has merely a function of accessing the recording medium210.

The connection unit 211 is an interface for connecting to an externaldevice. The smart device 200 can exchange data with an external devicevia the connection unit 211. In the present exemplary embodiment, theconnection unit 211 is an antenna, and the control unit 201 can beconnected to the digital camera 100 via the antenna. The connection unit211 may be directly connected to the digital camera 100 or may beconnected to the digital camera 100 via an access point. As a protocolfor data communication, Picture Transfer Protocol over Internet Protocol(PTP/IP) through a wireless LAN can be used, for example. Note that thecommunication with the digital camera 100 is not limited to this. Forexample, the connection unit 211 may be another wireless communicationmodule such as an infrared communication module, a Bluetooth®communication module, or a wireless USB.

A near field radio communication unit 212 is a communication unit forimplementing near field radio communication. The near field radiocommunication unit 212 is formed from an antenna for wirelesscommunication, a modulation/demodulation circuit for processing radiosignals, and a communication controller. The near field radiocommunication unit 212 outputs a modulated radio signal from the antennaand demodulates a radio signal received by the antenna, therebyimplementing near field radio communication. In this case, near fieldradio communication is implemented in conformity with the IEEE 802.15standard (Bluetooth®). Note that the non-contact proximity communicationimplemented by the near field radio communication unit 212 is notlimited to Bluetooth®, and other wireless communication may be adopted.

A public network connection unit 213 is an interface used for performingpublic wireless communication. The smart device 200 can make a telephonecall to another device via the public network connection unit 213. Atthis time, the control unit 201 implements the telephone call byinputting and outputting sound signals via a microphone 214 and aspeaker 215. In the present exemplary embodiment, the public networkconnection unit 213 is an antenna, and the control unit 201 can connectto the public network via the antenna. One antenna can serve as both theconnection unit 211 and the public network connection unit 213.

The smart device 200 has been described as an example of the electronicdevice in the present exemplary embodiment, but the electronic device isnot limited thereto. For example, the electronic device can be a digitalcamera with a wireless function or an information processing apparatussuch as a personal computer.

[About Image Composition]

Next, a process of combining a main image output from the imaging unit102 and a sub image received from the smart device 200 in the digitalcamera 100 will be described with reference to FIGS. 3, 4A, and 4B.

FIG. 3 is a sequence diagram illustrating processing by the digitalcamera 100 and the smart device 200 according to the first exemplaryembodiment.

FIG. 4A is a diagram illustrating an example of screen transition on thedisplay unit 106 at the time of combining the main image output from theimaging unit 102 and the sub image received from the smart device 200 inthe digital camera 100. FIG. 4B is a diagram illustrating an example inwhich, in the smart device 200, a shooting screen 470 is set based onthe shape of a wipe received from the digital camera 100 and displayedon the display unit 206.

In step S301, the control unit 101 of the digital camera 100 displays afunction selection screen 401 illustrated in FIG. 4A on the display unit106. On the function selection screen 401, a wipe shooting button 410for selecting the function of combining the main image and the sub imageis displayed. The user can select the function of combining the mainimage and the sub image by operating the operation unit 105 (forexample, the touch panel) and selecting the wipe shooting button 410. Onthe function selection screen 401, buttons for executing othercommunication functions are also displayed. For example, an imagetransfer button for selecting a function to transfer image data to anexternal device, a remote operation button for selecting a function toremotely operate the digital camera 100 from an external device aredisplayed.

Upon detecting the selection of the wipe shooting button 410, thecontrol unit 101 starts the processing in step S302 and subsequent stepsto performs a process for wireless LAN connection. For example, Wi-Fi®can be used as a wireless LAN standard.

In step S302, the control unit 101 of the digital camera 100 activatesthe simple AP to generate a wireless LAN network, and displays a screen402 illustrated in FIG. 4A on the display unit 106. On the screen 402,at least a service set identifier (SSID) as a network identifier isdisplayed so that peripheral devices can participate in the wireless LANnetwork. In addition, when the wireless LAN network is under securityusing an encryption key, the encryption key may also be displayed.

In step S303, the control unit 101 of the digital camera 100 startsdevice search. The device search can be performed by using a discoveryprotocol, for example, Single Service Discovery Protocol (SSDP),Malticast DNS, or the like.

In step S304, the control unit 201 of the smart device 200 performsnetwork selection. The control unit 201 displays an SSID list displayscreen (not illustrated) on the display unit 206 and accepts a networkselection of the simple AP generated by the digital camera 100 via theoperation unit 205. The SSID list display screen can be displayed byselecting an option for inputting an instruction to start wireless LANsetting from the OS menu. The display unit 106 of the digital camera 100displays the SSID of the network generated by the digital camera 100 asillustrated on the screen 402. The user can check the SSID displayed onthe screen 402 and select the SSID of the network generated by thedigital camera 100 from the SSID list displayed on the SSID list displayscreen.

When the network is selected by a user operation in step S304, the smartdevice 200 participates in the simple AP network of the digital camera100 in step S305. When the smart device 200 participates in the network,IP address and the like are set, and a connection between the digitalcamera 100 and the smart device 200 is established at a network level.

Then, a connection is established at an application level by thesubsequent processing. In step S306, the control unit 201 of the smartdevice 200 receives an instruction for activation of an applicationsaved in the non-volatile memory 203 via the operation unit 205.

In step S307, the control unit 201 of the smart device 200 broadcasts anadvertisement notification in the participating network according to thecontrol of the activated application to notify its own presence to thedigital camera 100. The advertisement notification can be made using adiscovery protocol, for example, SSDP, Malticast DNS, or the like.

In step S308, the control unit 101 of the digital camera 100 selects thedevice to be connected. Upon reception of the advertisement notificationin step S307, the control unit 101 displays a screen 403 illustrated inFIG. 4A on the display unit 106. On the screen 403, the device namesincluded in the advertisement notification are displayed to show a listof connectable devices to the user. The user can operate the operationunit 105 to select the device to connect from the screen 403. In thisexample, it is assumed that the smart device 200 is selected.

In step S309, the control unit 101 of the digital camera 100 selects thewipe shape of the device to connect. The control unit 101 displays awipe selection screen 404 illustrated in FIG. 4A on the display unit106. On the wipe selection screen 404, a plurality of wipe shapes ispresented in a selectable manner so that the user can operate theoperation unit 105 to select a desired wipe shape.

In step S310, the control unit 101 of the digital camera 100 starts aprocess of connecting with the smart device 200 as the device to connectselected in step S308. In conjunction with this, the control unit 101displays a screen 405 illustrated in FIG. 4A on the display unit 106.

In step S311, the control unit 201 of the smart device 200 notifies thedigital camera 100 of whether connection is enabled or disabled. Whenconnection is enabled, the smart device 200 notifies the digital camera100 that connection is OK, and when the connection is disabled, thesmart device 200 notifies the digital camera 100 that connection is NG.

In step S312, according to the notification in step S310, theapplication-level connection is completed between the digital camera 100and the smart device 200.

In step S313, the control unit 101 of the digital camera 100 determinesthe wipe shape. When there is a request for changing the wipe shape by auser operation, the control unit 101 displays the wipe selection screen404 illustrated in FIG. 4A on the display unit 106, so that the user canoperates the operation unit 105 to select and determine the desired wipeshape, When there is no request for changing the wipe shape, the wipeshape at the present time, that is, the wipe shape selected in step S309is maintained. When the wipe shape has been changed, the lastly changedwipe shape is maintained.

In step S314, the control unit 101 of the digital camera 100 transmits asub image request including the notification of the wipe shapedetermined in step S313 to the smart device 200.

In step S315, the control unit 201 of the smart device 200 transmits thesub image data to the digital camera 100. As illustrated in FIG. 4B, thecontrol unit 201 sets a shooting screen 470 for shooting by the imagingunit 202 and displays the same on the display unit 206. On the shootingscreen 470, an area for displaying the image captured by the imagingunit 202 is provided to match the wipe shape received in step S314, andthe other areas are blackened. In the present example, a horizontallylong elliptical wipe on the left middle of the wipe selection screen 404is determined. By setting the shooting screen according to the wipeshape in this way, the user can perform shooting with a consciousnessabout the wipe shape. Then, the control unit 201 transmits the imagedata output from the imaging unit 202 and adjusted based on the wipeshape as sub image data to the digital camera 100. Adjusting the imagedata based on the wipe shape refers to cutting out the image from theimaging unit 202 according to the wipe shape, for example.

In step S316, the control unit 101 of the digital camera 100 combinesthe main image output from the imaging unit 102 and the sub imagereceived in step S315. As a result, the display unit 106 of the digitalcamera 100 displays the composite image such that the sub image issuperimposed on the main image and is displayed in a wipe 460 asillustrated on the screen 406 in FIG. 4A. When shooting is performed inthis state, the composite image data may he saved in the cording medium110.

The digital camera 100 and the smart device 200 repeatedly execute theprocessing in steps S313 to S316.

As described above, in the digital camera 100, the main image outputfrom the imaging unit 102 and the sub image received from the smartdevice 200 can be combined.

The wipes may be prepared in the same shape and different sizes, forexample. In this case, even if wipe shapes of different sizes areselected, the wipes are regarded as having no change in shape.

[Operations of the Digital Camera 100]

Next, the operations of the digital camera 100 for implementing theforegoing procedure will be described.

FIG. 5A is a flowchart illustrating the operations of the digital camera100.

In step S501, the control unit 101 determines whether the function ofcombining the main image and the sub image has been selected. Theprocessing in this step corresponds to step S301 of FIG. 3. When thecontrol unit 101 determines that the image composition function has beenselected, the process proceeds to step S502. Otherwise, the control unit101 waits until the image composition function is selected.

In step S502, the control unit 101 activates the simple AP to generate awireless LAN network. The processing in this step corresponds to stepS302 of FIG. 3.

In step S503, the control unit 101 starts device search. The processingin this step corresponds to step S303 of FIG. 3.

In step S504, the control unit 101 determines whether any device hasbeen discovered. When receiving the advertisement notification in stepS307 of FIG. 3, the control unit 101 determines that any device has beendiscovered. When receiving no advertisement notification, the controlunit 101 determines that no device has been discovered. In this step, itmay be determined that no device has been discovered when noadvertisement notification has been received for a predetermined time.When the control unit 101 determines that any device has beendiscovered, the process proceeds to step S505. Otherwise, the processexits from this flow.

In step S505, the control unit 101 selects the device to connect. Theprocessing in this step corresponds to step S308 of FIG. 3. At theselection of the device to connect, the selection by the user operationis accepted in step S308. Alternatively, the device to connect may bedetermined by selecting the device satisfying a predetermined conditionsuch as the firstly discovered device. In this example, it is assumedthat the smart device 200 is selected.

In step S506, the control unit 101 selects the wipe shape for the deviceto connect. The processing in this step corresponds to step S309 of FIG.3. In step S309, the selection by a user operation is accepted.Alternatively, the default wipe shape may be selected, or the wipe shapeselected at the previous connection to the device to connect may beautomatically selected.

In step S507, the control unit 101 performs a process for connectingwith the smart device 200 as the device to connect selected in stepS505. The processing in this step corresponds to steps S310 to S312 ofFIG. 3.

In step S508, the control unit 101 determines the wipe shape. Theprocessing in this step corresponds to step S313 of FIG. 3.

The wipe shape determination processing in step S508 will be describedin detail with reference to FIG. 5B.

In step S551, the control unit 101 determines whether there is a requestfor changing the wipe shape by a user operation. When the control unit101 determines that there is a request for changing the wipe shape, theprocess proceeds to step S552. Otherwise, the process proceeds to stepS555.

In step S552, the control unit 101 displays the wipe selection screen404 illustrated in FIG. 4A on the display unit 106.

In step S553, the control unit 101 determines whether the wipe shape hasbeen changed by a user operation. When the control unit 101 determinesthat the wipe shape has been changed, the process proceeds to step S554.Otherwise, the process proceeds to step S555.

In step S554, the control unit 101 determines to change the wipe shape.In step S555, the control unit 101 determines to maintain the wipeshape. Accordingly, the wipe shape determination processing isterminated.

Returning to FIG. 5A, in step S509, the control unit 101 transmits a subimage request including the notification of the wipe shape determined instep S508 to the smart device 200. The processing in this stepcorresponds to step S314 of FIG. 3.

In step S510, the control unit 101 receives sub image data from smartdevice 200 as a response to the sub image request transmitted to thesmart device 200 in step S509. In this case, the sub image received hereis an image in the wipe shape determined in step S508. The processing instep S510 corresponds to step S315 of FIG. 3.

In step S511, the control unit 101 combines the main image output fromthe imaging unit 102 and the sub image received in step S510. Theprocessing in step S511 corresponds to step S316 of FIG. 3.

In step S512, the control unit 101 determines whether to terminate theprocess. Terminating the process refers to terminating the compositionof the main image and the sub image, or powering off and shutting downthe digital camera 100. When the control unit 101 determines not toterminate the process, the process returns to step S508. Otherwise, theprocess exits from this flow.

As described above, when the main image and the sub image are combined,the wipe shape of the sub image can be changed. In the present exemplaryembodiment, in the digital camera 100, when the main image output fromthe imaging unit 102 and the sub image received from the smart device200 are combined, the wipe shape is determined on the digital camera 100side. The digital camera 100 notifies the smart device 200 of the wipeshape, and the smart device 200 transmits to the digital camera 100 theimage data of the notified wipe shape as sub image data. As a result, itis possible to combine the main image and the sub image in the wipeshape desired by the user.

Second Exemplary Embodiment

Next, a second exemplary embodiment will be described. The configurationof a digital camera 100 is the same as that of the digital camera 100according to the first exemplary embodiment, and descriptions thereofwill be omitted. In the present exemplary embodiment, two smart devicesA and B are taken, both of which are the same as the smart device 200according to the first exemplary embodiment, and thus descriptionsthereof will be omitted.

[About Image Composition]

A process of combining a main image output from an imaging unit 102, asub image received from the smart device A, and a sub image receivedfrom the smart device B in the digital camera 100 will be described withreference to FIGS. 6 and 7.

FIG. 6 is a sequence diagram illustrating processing by the digitalcamera 100 and the smart devices A and B according to the secondexemplary embodiment.

FIG. 7A illustrates an example of screen transition displayed on thedisplay unit 106 when the main image output from the imaging unit 102,the sub image received from the smart device A, and the sub imagereceived from the smart device B are combined in the digital camera 100.FIG. 7B is a diagram illustrating an example of a wipe selection screen708 in the smart devices A and B. FIG. 7C is a diagram illustrating anexample in which, in the smart devices A and B, a shooting screen 770 isset based on the determined wipe shape and is displayed on the displayunit 206.

In step S601, as in step S301 of FIG. 3, the control unit 101 of thedigital camera 100 displays a function selection screen 701 illustratedin FIG. 7A on the display unit 106. On the function selection screen701, a wipe shooting button 710 for selecting the function of combiningthe main image and the sub image is displayed. The user can select thefunction of combining the main image and the sub image by operating theoperation unit 105 (for example, the touch panel) and selecting the wipeshooting button 710.

Upon detecting the selection of the wipe shooting button 710, thecontrol unit 101 starts the processing in step S602 and subsequent stepsto perform a process for wireless LAN connection.

In step S602, as in step S302 of FIG. 3, the control unit 101 of thedigital camera 100 activates the simple AP to generate a wireless LANnetwork and displays a screen 702 illustrated in FIG. 7A on the displayunit 106.

In step S603, as in step S303 of FIG. 3, the control unit 101 of thedigital camera 100 starts device search.

In step S604, as in step S304 of FIG. 3, the control unit 201 of thesmart device A performs network selection.

When the network is selected by the user operation in step S604, thesmart device A participates in the simple AP network of the digitalcamera 100 in step S605 as in step S305 of FIG. 3.

Then, a connection is established at an application level by thesubsequent processing. In step S606, as in step S306 of FIG. 3, thecontrol unit 201 of the smart device A receives an instruction foractivation of an application saved in the non-volatile memory 203 viathe operation unit 205.

In step S607, as in step S307 of FIG. 3, the control unit 201 of thesmart device A broadcasts an advertisement notification to theparticipating network according to the control of the activatedapplication to notify its own presence to the digital camera 100.

In steps S608 to S611, the smart device B also executes the sameprocessing as in the steps S604 to S607.

In step S612, as in step S308 of FIG. 3, the control unit 101 of thedigital camera 100 selects the device to connect. Upon reception of theadvertisement notification in step S607 or step S611, the control unit101 displays a screen 703 illustrated in FIG. 7A on the display unit106. On the screen 703, the device names included in the advertisementnotification are displayed to show a list of connectable devices to theuser. The user can operate the operation unit 105 to select the deviceto connect from the screen 703. In the present example, it is assumedthat the smart device A is selected.

In step S613, as in step S310 of FIG. 3, the control unit 101 of thedigital camera 100 starts a process of connecting with the smart deviceA as the device to connect selected in step S612. In conjunction withthis, the control unit 101 displays a screen 704 illustrated in FIG. 7Aon the display unit 106.

In step S614, as in step S311 of FIG. 3, the control unit 201 of thesmart device A notifies the digital camera 100 of whether connection isenabled or disabled.

In step S615, as in step S312 of FIG. 3, according to the notificationin step S613, the application-level connection between the digitalcamera 100 and the smart device A is completed.

In step S616, for further connection with another external device, thecontrol unit 101 of the digital camera 100 selects the device to connectas in step S612. The control unit 101 displays a screen 705 illustratedin FIG. 7A on the display unit 106. On the screen 705, the device namesincluded in the advertisement notification are displayed to show a listof connectable devices to the user. On the screen 705, as compared withthe screen 703, the device name of the smart device A already selectedas the device to connect is deleted. The user can operate the operationunit 105 to select the device to connect from the screen 705. In thisexample, it is assumed that the smart device B is selected.

Steps S617 to S619 are the same as steps S613 to S615, and theapplication-level connection between the digital camera 100 and thesmart device 13 is completed. When starting the process of connectionwith the smart device b, the control unit 101 displays a screen 706illustrated in FIG. 7A on the display unit 106.

In step S620, the control unit 101 of the digital camera 100 transmits asub image request to the smart device A.

In step S621, the control unit 101 of the digital camera 100 transmitsthe sub image request to the smart device B.

In step S622, the control unit 201 of the smart device A determines thewipe shape. When there is a request for changing the wipe shape by auser operation, the control unit 201 displays a wipe selection screen708 illustrated in FIG. 79 on the display unit 206, so that the user canoperate the operation unit 205 to select and determine the desired wipeshape. When there is no request for changing the wipe shape, the wipeshape at the present time is maintained.

In step S623, the control unit 201 of the smart device A transmits thesub image data to the digital camera 100. As illustrated in FIG. 7C, thecontrol unit 201 sets a shooting screen 770 for shooting by the imagingunit 202 and displays the same on the display unit 206. On the shootingscreen 770, an area for displaying the image captured by the imagingunit 202 is provided to match the wipe shape determined in step S622,and the other areas are blackened. In this example, a vertically longrectangular wipe at the upper right corner of the wipe selection screen708 is determined. By setting the shooting screen according to the wipeshape in this way, the user can perform shooting with a consciousnessabout the wipe shape. Then, the control unit 201 transmits the imagedata output from the imaging unit 202 and adjusted based on the wipeshape as sub image data to the digital camera 100. Adjusting the imagedata based on the wipe shape refers to cutting out the image from theimaging unit 202 according to the wipe shape, for example.

In steps S624 and S625, the smart device B also executes the sameprocessing as in the steps S622 and S623.

In step S626, the control unit 101 of the digital camera 100 combinesthe main image output from the imaging unit 102, the sub image receivedin step S623, and the sub image received in step S625. Accordingly, thedisplay unit 106 of the digital camera 100 displays the composite imagesuch that the sub image received from the smart device A is superimposedon the main image and displayed in a wipe 760, and the sub imagereceived from the smart device B is superimposed on the main image anddisplayed in a wipe 761 as illustrated on a screen 707 in FIG. 7A. Whenshooting is performed in this state, the composite image data may besaved in the recording medium 110.

The digital camera 100 and the smart devices A and B repeatedly executethe processing in steps S622 to S626.

As described above, the main image output from the imaging unit 102, thesub image received from the smart device A, and the sub image receivedfrom the smart device B can be combined in the digital camera 100.

[Operations of the Digital Camera 100]

Next, the operations of the digital camera 100 for implementing theforegoing procedure will be described.

FIG. 8 is a flowchart describing the operations of the digital camera100.

Steps S801 to S806 are the same as steps S501 to S505 and S507 of FIG.5A, and descriptions thereof will be omitted here.

In step S807, the control unit 101 determines whether to performconnection to another external device. In this example, it is assumedthat the digital camera 100 connects to the smart device A, and thenconnects to the smart device B. Whether to perform connection to anotherexternal device may be selected by a user operation, or the number ofdevices to be connected may be predetermined in advance, for example.When the control unit 101 determines to perform connection to anotherexternal device, the process returns to step S805. Otherwise, theprocess proceeds to step S808.

In step S808, the control unit 101 transmits a sub image request to thesmart device A and the smart device B. The processing in this stepcorresponds to steps S620 and S621 of FIG. 6.

In step S809, the control unit 101 receives the sub image data from thesmart device A and the smart device B as a response to the sub imagerequest transmitted to the smart device A and the smart device B in stepS808. The sub images received here are images in the wipe shapesdetermined in the smart device A and the smart device B. The processingin step S809 corresponds to steps S623 and S625 of FIG. 6.

In step S810, the control unit 101 combines the main image output fromthe imaging unit 102 and the sub images received in step S809. Theprocessing in step S810 corresponds to step S626 of FIG. 6.

In step S811, the control unit 101 determines whether to terminate theprocess. Terminating the process refers to terminating the compositionof the main image and the sub image, or powering off and shutting downthe digital camera 100. When the control unit 101 determines not toterminate the process, the process returns to step S809. Otherwise, theprocess exits from this flow.

[Operations of the Smart Device 200]

Next, the operations of the smart device 200 (the smart devices A and B)for implementing the foregoing procedure will be described.

FIG. 9A is a flowchart describing the operations of the smart device200.

In step S901, the control unit 201 determines whether a network has beenselected by the user. The processing in this step corresponds to stepsS604 and S608 of FIG. 6. When the control unit 201 determines that anetwork has been selected, the process proceeds to step S902. Otherwise,the control unit 201 waits until a network is selected.

In step S902, the control unit 201 participates in the wireless LANnetwork selected in step S901. The processing in this step correspondsto step S605 or S609 of FIG. 6.

In step S903, the control unit 201 determines whether the applicationhas been activated by a user operation. The processing in this stepcorresponds to step S606 or S610 of FIG. 6. When the control unit 201determines that the application has been activated, the process proceedsto step S904. Otherwise, the control unit 201 waits until theapplication is activated.

In step S904, the control unit 201 broadcasts an advertisementnotification to the participating network according to the control ofthe activated application to notify its own presence to the digitalcamera 100. The processing in this step corresponds to step S607 or S611of FIG. 6.

In step S905, the control unit 201 performs a process of connection withthe digital camera 100. The processing in this step corresponds to stepsS613 to S615 or steps S617 to S619 of FIG. 6.

In step S906, the control unit 201 receives a sub image request from thedigital camera 100. The processing in this step corresponds to step S620or S621 of FIG. 6.

In step S907, the control unit 201 determines the wipe shape. Theprocessing in this step corresponds to step S622 or S624 of FIG. 6.

The wipe shape determination processing in step S907 will be describedin detail with reference to FIG. 9B.

In step S951, the control unit 201 determines whether there is a requestfor changing the wipe shape by a user operation. When the control unit201 determines that there is a request for changing the wipe shape, theprocess proceeds to step S952. Otherwise, the process proceeds to stepS955.

In step S952, the control unit 201 displays the wipe selection screen708 illustrated in FIG. 7B on the display unit 206.

In step S953, the control unit 201 determines whether the wipe shape hasbeen changed by a user operation. When the control unit 201 determinesthat the wipe shape has been changed, the process proceeds to step S954.Otherwise, the process proceeds to step S955.

In step S954, the control unit 201 determines to change the wipe shape.In step S955, the control unit 201 determines to maintain the wipeshape. Accordingly, the wipe shape determination processing isterminated.

Returning to the description of FIG. 9A, in step S908, the control unit201 transmits the sub image data to the digital camera 100. Theprocessing in this step corresponds to step S623 or S625 of FIG. 6.

In step S909, the control unit 201 determines whether to terminate theprocess. Terminating the process refers to terminating the compositionof the main image and the sub image, or powering off and shutting downthe smart device 200. When the control unit 201 determines not toterminate the process, the process returns to step S907. Otherwise, theprocess exits from this flow.

As described above, when the main image and the sub image are combined,the wipe shape of the sub image can be changed. In the present exemplaryembodiment, in the digital camera 100, when the main image output fromthe imaging unit 102 and the sub image received from the smart device200 (the smart devices A and B) are combined, the wipe shape isdetermined on the smart device 200 side. The smart device 200 transmitsthe image data on the wipe shape determined by itself as sub image datato the digital camera 100. As a result, the main image and the sub imagecan be combined in the wipe shape desired by the user.

In the first and second exemplary embodiments described above, the wipeshape can be changed. In addition thereto, the size of the wipe and theposition of the wipe with respect to the main image may be changeable.

The above-described exemplary embodiments are merely examples and arenot seen to be limiting. That is, aspects of the present disclosure canbe carried out in various forms without departing from the technicalidea or the main features thereof. in addition, the exemplaryembodiments can be combined as appropriate.

Other Embodiments

Embodiment(s) can also be realized by a computer of a system orapparatus that reads out and executes computer executable instructions(e.g., one or more programs) recorded on a storage medium (which mayalso be referred to more fully as a ‘non-transitory computer-readablestorage medium’) to perform the functions of one or more of theabove-described embodiment(s) and/or that includes one or more circuits(e.g., application specific integrated circuit (ASIC)) for performingthe functions of one or more of the above-described embodiment(s), andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s) and/or controlling the one or morecircuits to perform the functions of one or more of the above-describedembodiment(s). The computer may comprise one or more processors (e.g.,central processing unit (CPU), micro processing unit (MPU)) and mayinclude a network of separate computers or separate processors to readout and execute the computer executable instructions. The computerexecutable instructions may be provided to the computer, for example,from a network or the storage medium. The storage medium ay include, forexample, one or more of a hard disk, a random-access memory (RAM), aread only memory (ROM), a storage of distributed computing systems, anoptical disk (such as a compact disc (CD), digital versatile disc (DVD),or Blu-ray Disc (BD)™), a flash memory device, a memory card, and thelike.

While exemplary embodiments have been described, it is to be understoodthat the disclosure is not limited to the disclosed exemplaryembodiments. The scope of the following claims is to be accorded thebroadest interpretation so as to encompass all such modifications andequivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2018-086423, filed Apr. 27, 2018, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image processing apparatus that communicateswith an external device and combines a sub image received from theexternal device with a main image, the image processing apparatuscomprising: a determination unit configured to determine a wipe shape ofthe sub image; and a notification unit configured to notify the externaldevice of the determined wipe shape.
 2. The image processing apparatusaccording to claim 1, wherein an image output from an imaging unit isset as the main image.
 3. The image processing apparatus according toclaim 1, a plurality of wipe shapes are presented in a selectable mannerand the determination unit determines the wipe shape via a selection bya user operation.
 4. The image processing apparatus according to claim1, wherein the notification unit makes a sub image request to theexternal device, wherein the sub image request includes a notificationof the determined wipe shape.
 5. An electronic device that communicateswith an image processing apparatus that combines a sub image with a mainimage, the electronic device comprising: a connection unit configured toreceive a notification of a wipe shape of the sub image from the imageprocessing apparatus; and a control unit configured to perform controlsuch that data of the sub image is transmitted to the image processingapparatus.
 6. The electronic device according to claim 5, wherein animage output from an imaging unit is the sub image.
 7. The electronicdevice according to claim 6, further comprising a setting unitconfigured to set a shooting screen for shooting by the imaging unitbased on the wipe shape of which notification is received by the unitconfigured to receive the notification.
 8. The electronic deviceaccording to claim 6, wherein the control unit performs control suchthat the image output from the imaging unit is adjusted based on thereceived notification of wipe shape.
 9. An electronic device thatcommunicates with an image processing apparatus that combines a subimage with a main image, the electronic device comprising: adetermination unit configured to determine a wipe shape of the subimage; and a control unit configured to perform control such that dataof the sub image is transmitted to the image processing apparatus. 10.The electronic device according to claim 9, wherein an image output froman imaging unit is the sub image.
 11. The electronic device according toclaim 10, further comprising a setting unit configured to set a shootingscreen for shooting by the imaging unit based on the determined wipeshape.
 12. The electronic device according to claim 10, wherein thecontrol unit performs control such that the image output from theimaging unit is adjusted based on the determined wipe shape and istransmitted as the sub image.
 13. A control method of an imageprocessing apparatus that communicates with an external device andcombines a sub image received from the external device with a mainimage, the control method comprising: determining a wipe shape of thesub image; and notifying the external device of the determined wipeshape.
 14. A control method of an electronic device that communicateswith an image processing apparatus that combines a sub image with a mainimage, the control method comprising: receiving a notification of a wipeshape of the sub image from the image processing apparatus; andtransmitting data of the sub image to the image processing apparatus.15. A control method of an electronic device that communicates with animage processing apparatus that combines a sub image with a main image,the control method comprising: determining a wipe shape of the subimage; and transmitting data of the sub image to the image processingapparatus.
 16. A non-transitory computer-readable storage medium thatstores a program for causing a computer to execute a control method ofan image processing apparatus that communicates with an external deviceand combines a sub image received from the external device with a mainimage, the control method comprising: determining a wipe shape of thesub image; and notifying the external device of the determined wipeshape.
 17. A non-transitory computer-readable storage medium storing aprogram for causing a computer to execute a control method of anelectronic device that communicates with an image processing apparatusthat combines a sub image with a main image, the control methodcomprising: receiving a notification of a wipe shape of the sub imagefrom the image processing apparatus; and transmitting data of the subimage to the image processing apparatus.